Measuring the "whole system" outcomes of an educational innovation: experience from the integrative family medicine program.

BACKGROUND AND OBJECTIVES Six family medicine residency programs in the United States collaborated on the development and implementation of an integrative family medicine (IFM) program, which is a postgraduate training model that combines family medicine residency training with an integrative medicine fellowship. This paper reports on effects of IFM on residency programs and clinical systems in which it was implemented. METHODS We used the Integrative Medicine Attitudes Questionnaire (IMAQ) to assess participants' attitudes toward integrative medicine before and after the program was implemented. We assessed residency program recruitment success before and after the program was implemented. We conducted interviews with key informants at each program to evaluate the effects of the IFM on the six participating residency programs. RESULTS IMAQ scores demonstrated a significant increase in the acceptance of integrative medicine after implementation of IFM. Recruiting data showed that participating programs filled at a rate consistently above the national average both before and after implementation. Analysis of interview data showed that programs became more open to an integrative medicine (IM) approach and offered a wider range of clinical services to patients. CONCLUSIONS Our mixed-methods strategy for evaluation of IFM showed that implementing the program increased acceptance of IM, did not affect residency fill rates, and increased use of IM in clinical practice. The combination of quantitative and qualitative methods was an effective strategy for documenting the "systems level" effects of a new educational program.